Calibration system for calibrating visual coordinate system and depth coordinate system, calibration method and calibration device

ABSTRACT

The disclosure provides a calibration system, a calibration method, and a calibration device. The calibration method for obtaining a transformation of coordinate systems between a vision sensor and a depth sensor includes the following steps. (a) A first coordinate group of four endpoints of a calibration board in a world coordinate system is created. (b) An image of the calibration board is obtained by the vision sensor, and a second coordinate group of the four endpoints of the calibration board in a two-dimensional coordinate system is created. (c) A third coordinate group of the four endpoints of the calibration board in a three-dimensional coordinate system is created according to the first and second coordinate groups. (d) The third coordinate group is transformed to a fourth coordinate group corresponding to the depth sensor to obtain the transformation of the coordinate systems according to at least three target scanning spots.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Taiwan applicationserial no. 109121082, filed on Jun. 22, 2020. The entirety of theabove-mentioned patent application is hereby incorporated by referenceherein and made a part of this specification.

BACKGROUND Technical Field

The disclosure relates to image calibration technology, and inparticular, to a calibration system for calibrating a visual coordinatesystem and a depth coordinate system, a calibration method, and acalibration device configured for the transformation of coordinatesystems between a vision sensor and a depth sensor.

Description of Related Art

Nowadays, self-driving technologies applied in self-driving cars arebooming. Therefore, vision sensors and depth sensors are widely appliedto replace the human perception, recognition, and positioning withrespect to the environment in driving. Common vision sensors are, forexample, color cameras, monochrome cameras, and near-infrared cameras.The vision sensor exhibits low cost and resemblance to human vision, soit is well adapted to object recognition. However, the vision sensor iseasily affected by the environment and the light source, leading to anunstable detection distance. Common depth sensors adopt rangingtechnologies such as structured light, time of flight (ToF), lightdetection and ranging (LiDAR, i.e., light radar or laser radar),millimeter wave radar, and other ranging technologies. Compared to thevision sensor, the depth sensor is less susceptible to the effect ofweather, environment, and light source. Therefore, the depth sensor cancollect stable depth information and has advantages in simultaneouslocalization and mapping (SLAM), real-time obstacle avoidance, andthree-dimensional object recognition. However, the cost of the depthsensor is relatively high. Accordingly, the vision sensor and the depthsensor are applied at the same time to complement each other in thepractical application.

Based on the above, in the practical application of the vision sensorand the depth sensor, the vision sensor and the depth sensor both formcoordinate systems based on their respective origins. Therefore, tointegrate the vision sensor and the depth sensor, it is necessary tocalibrate the relationship between the two coordinate systems, that is,to obtain a transformation of coordinate system in which the visionsensor coordinate system is transformed to the depth sensor coordinatesystem or the depth sensor coordinate system is transformed to thevision sensor coordinate system. The transformation of coordinatesystems, also called an extrinsic parameters, includes rotation andtranslation.

In order to accurately obtain the transformation between the visionsensor coordinate system and the depth sensor coordinate system, methodsfor calibrating the extrinsic parameters have been proposed.Conventional and current methods can be roughly categorized into methodsusing a target object and methods not using a target object.

In the case of the methods using a target object, a checkerboardcalibration board, for example, may be used to calibrate the extrinsicparameters between the single-line laser radar and the vision sensor(e.g., a camera). The normal vectors of the checkerboard calibrationboard in a visual coordinate system (camera coordinate system) iscalculated through the checkerboard calibration board. The normalvectors and the laser points incident on the checkerboard calibrationboard form geometric constraints which may be used as extrinsicparameters, and then the extrinsic parameters are solved by collectingthe postures of multiple checkerboard calibration boards. The aboveposture may include, for example, a position and a pose of thecheckerboard calibration board. In addition, in other methods, differenttarget objects, such as boxes, round holes, or trihedrons, are used forcalibration. However, there are still many limitations in the practicalapplication of the methods using specific target objects. That is, thetarget object itself should be easily recognized by the vision sensor,and the range of the target object itself must be large enough and wellplaced for both the vision sensor and the depth sensor to detect.Moreover, the target object should be easily manufactured; otherwise, itis difficult to apply the methods using such target objects.

In the case of the methods not using a target object, for example,without relying on any scene or target object, only by using mutualinformation between the intensity of a radar echo and the grayscale of acamera image, the extrinsic parameters between the vision sensor and thedepth sensor are continuously optimized starting from the initial valueof the extrinsic parameters. However, there is a blind spot in theimplementation of the method, so the initial value of the extrinsicparameters may greatly affect the final result and it is not easy toestimate the initial value of the extrinsic parameters. In addition,with the inconsistency in the standard used in shooting the scene forcalibration, the intensity of radar echo of different object materialsis different and has no absolute relationship with the grayscale in thevision sensor, so the calibration may eventually fail to converge.

The information disclosed in this Background section is only forenhancement of understanding of the background of the describedtechnology and therefore it may contain information that does not formthe prior art that is already known to a person of ordinary skill in theart. Further, the information disclosed in the Background section doesnot mean that one or more problems to be resolved by one or moreembodiments of the invention were acknowledged by a person of ordinaryskill in the art.

SUMMARY

In view of this, the disclosure provides a calibration method, acalibration device, and a calibration system for calibrating a visualcoordinate system and a depth coordinate system, which can be used tosolve the above problems.

The disclosure provides a calibration method for obtaining atransformation of coordinate systems between a vision sensor and a depthsensor. The calibration method includes the following steps. (a) A firstcoordinate group of four endpoints of a calibration board in a worldcoordinate system is created. (b) An image of the calibration board isobtained by the vision sensor and a second coordinate group of the fourendpoints of the calibration board in a two-dimensional coordinatesystem is created. (c) A third coordinate group of the four endpoints ofthe calibration board corresponding to the vision sensor in athree-dimensional coordinate system is created according to the firstcoordinate group and the second coordinate group. (d) The thirdcoordinate group is transformed to a fourth coordinate groupcorresponding to the depth sensor to obtain the transformation of thecoordinate system according to at least three target scanning spotsgenerated by the depth sensor.

In an embodiment of the disclosure, after the step of creating thesecond coordinate group of the four endpoints of the calibration boardin the two-dimensional coordinate system, the calibration method furtherincludes the following step. A first transformation between the firstcoordinate group and the second coordinate group is obtained accordingto a geometric correspondence between the first coordinate group and thesecond coordinate group and an intrinsic parameter of the vision sensor.The step of creating the third coordinate group of the four endpoints ofthe calibration board in the three-dimensional coordinate systemaccording to the first coordinate group and the second coordinate groupincludes the following step. The first coordinate group is transformedto the third coordinate group according to the first transformation.

In an embodiment of the disclosure, the step of transforming the thirdcoordinate group to the fourth coordinate group corresponding to thedepth sensor includes the following step. The third coordinate group istransformed to the fourth coordinate group according to a secondtransformation. The second transformation is a correspondence between acoordinate system of the vision sensor and a coordinate system of thedepth sensor.

In an embodiment of the disclosure, the at least three target scanningspots are located in a quadrilateral formed by the four endpoints in thefourth coordinate group.

In an embodiment of the disclosure, the step of transforming the thirdcoordinate group to the fourth coordinate group corresponding to thedepth sensor to obtain the transformation of coordinate systemsaccording to the at least three target scanning spots generated by thedepth sensor includes the following steps. A sum of areas of fourtriangles formed by scanning spots generated by the depth sensor and thefour endpoints in the fourth coordinate group is calculated. The secondtransformation is solved by taking, as the target scanning spots,scanning spots of which a difference between the sum of the areas of thefour triangles and an area of the quadrilateral is less than an errorvalue.

In an embodiment of the disclosure, the calibration board is a flatplate, a posture of the calibration board is within a visible range ofthe vision sensor and the depth sensor, and the calibration methodfurther includes the following steps. (e) Another posture of thecalibration board is obtained and it is determined whether a posturecount value is greater than or equal to a target value. If the posturecount value is greater than or equal to the target value, step (f) isperformed, and if the posture count value is less than the target value,the posture count value is counted, and step (a) is re-performed on thecalibration board with the another posture. (f) The transformation ofcoordinate systems is solved with a plurality of target scanning spotscorresponding to postures in a quantity of the target value.

In an embodiment of the disclosure, the transformation of coordinatesystems includes a correspondence, a rotation angle, and a translationbetween a coordinate system of the vision sensor and a coordinate systemof the depth sensor.

The disclosure provides a calibration system for calibrating a visualcoordinate system and a depth coordinate system. The calibration systemincludes a vision sensor, a depth sensor, and a processor. The visionsensor is configured to obtain an image of a calibration board, thedepth sensor is configured to generate a plurality of scanning spots,and the processor is coupled to the vision sensor and the depth sensorto obtain a transformation of coordinate system between the visionsensor and the depth sensor. The processor is configured to (a) create afirst coordinate group of four endpoints of the calibration board in aworld coordinate system; (b) create a second coordinate group of thefour endpoints of the calibration board in a two-dimensional coordinatesystem according to the image of the calibration board; (c) create athird coordinate group of the four endpoints of the calibration boardcorresponding to the vision sensor in a three-dimensional coordinatesystem according to the first coordinate group and the second coordinategroup; and (d) transform the third coordinate group to a fourthcoordinate group corresponding to the depth sensor to obtain thetransformation of coordinate systems according to at least three targetscanning spots.

In an embodiment of the disclosure, after the operation of creating thesecond coordinate group of the four endpoints of the calibration boardin the two-dimensional coordinate system, the processor is configured toobtain a first transformation between the first coordinate group and thesecond coordinate group according to a geometric correspondence betweenthe first coordinate group and the second coordinate group and anintrinsic parameter of the vision sensor. In the operation of creatingthe third coordinate group of the four endpoints of the calibrationboard in the three-dimensional coordinate system according to the firstcoordinate group and the second coordinate group, the processor isconfigured to transform the first coordinate group to the thirdcoordinate group according to the first transformation.

In an embodiment of the disclosure, in the operation of transforming thethird coordinate group to the fourth coordinate group corresponding tothe depth sensor, the processor is configured to transform the thirdcoordinate group to the fourth coordinate group according to a secondtransformation. The second transformation is a correspondence between acoordinate system of the vision sensor and a coordinate system of thedepth sensor.

In an embodiment of the disclosure, the at least three target scanningspots are located in a quadrilateral formed by the four endpoints in thefourth coordinate group.

In an embodiment of the disclosure, in the operation of transforming thethird coordinate group to the fourth coordinate group corresponding tothe depth sensor to obtain the transformation of coordinate systemsaccording to the at least three target scanning spots generated by thedepth senor on the calibration board, the processor is configured tocalculate a sum of areas of four triangles formed by the scanning spotsand the four endpoints in the fourth coordinate group and to solve thesecond transformation by taking, as the target scanning spots, scanningspots of which a difference between the sum of the areas of the fourtriangles and an area of the quadrilateral is less than an error value.

In an embodiment of the disclosure, the calibration board is a flatplate, and a posture of the calibration board is within a visible rangeof the vision sensor and the depth sensor. The processor is furtherconfigured to (e) obtain another posture of the calibration board anddetermine whether a posture count value is greater than or equal to atarget value. If the posture count value is greater than or equal to thetarget value, operation (f) is performed, and if the posture count valueis less than the target value, the posture count value is counted, andoperation (a) is re-performed on the calibration board with the anotherposture. The processor is configured to (f) solve the transformation ofcoordinate systems with a plurality of target scanning spotscorresponding to postures in a quantity of the target value.

In an embodiment of the disclosure, the transformation of coordinatesystems includes a correspondence, a rotation angle, and a translationbetween a coordinate system of the vision sensor and a coordinate systemof the depth sensor.

The disclosure provides a calibration device including a storage circuitand a processor. The storage circuit stores images of a calibrationboard obtained by a vision sensor and stores a plurality of scanningspots generated by a depth sensor on the calibration board. Theprocessor is coupled to the storage circuit and accesses the images andthe scanning spots to (a) create a first coordinate group of fourendpoints of the calibration board in a world coordinate system; (b)create a second coordinate group of the four endpoints of thecalibration board in a two-dimensional coordinate system according tothe images of the calibration board; (c) create a third coordinate groupof the four endpoints of the calibration board corresponding to thevision sensor in a three-dimensional coordinate system according to thefirst coordinate group and the second coordinate group; and (d)transform the third coordinate group to a fourth coordinate groupcorresponding to a depth sensor to obtain a transformation of coordinatesystem according to at least three target scanning spots.

In an embodiment of the disclosure, after the operation of creating thesecond coordinate group of the four endpoints of the calibration boardin the two-dimensional coordinate system, the processor is configured toobtain a first transformation between the first coordinate group and thesecond coordinate group according to a geometric correspondence betweenthe first coordinate group and the second coordinate group and anintrinsic parameter of the vision sensor. In the operation of creatingthe third coordinate group of the four endpoints of the calibrationboard in the three-dimensional coordinate system according to the firstcoordinate group and the second coordinate group, the processor isconfigured to transform the first coordinate group to the thirdcoordinate group according to the first transformation.

In an embodiment of the disclosure, in the operation of transforming thethird coordinate group to the fourth coordinate group corresponding tothe depth sensor, the processor is configured to transform the thirdcoordinate group to the fourth coordinate group according to a secondtransformation. The second transformation is a correspondence between acoordinate system of the vision sensor and a coordinate system of thedepth sensor.

In an embodiment of the disclosure, the at least three target scanningspots are located in a quadrilateral formed by the four endpoints in thefourth coordinate group.

In an embodiment of the disclosure, in the operation of transforming thethird coordinate group to the fourth coordinate group corresponding tothe depth sensor to obtain the transformation of coordinate systemsaccording to the at least three target scanning spots generated by thedepth sensor on the calibration board, the processor is configured tocalculate a sum of areas of four triangles formed by the target scanningspots and the four endpoints in the fourth coordinate group and to solvethe second transformation by taking, as the target scanning spots,scanning spots of which a difference between the sum of the areas of thefour triangles and the area of the quadrilateral is less than an errorvalue.

In an embodiment of the disclosure, the calibration board is a flatplate, and a posture of the calibration board is within a visible rangeof the vision sensor and the depth sensor. The processor is furtherconfigured to (e) obtain another posture of the calibration board anddetermine whether a posture count value is greater than or equal to atarget value. If the posture count value is greater than or equal to thetarget value, operation (f) is performed, and if the posture count valueis less than the target value, the posture count value is counted, andoperation (a) is re-performed on the calibration board with the anotherposture. The processor is configured to (f) solve the transformation ofcoordinate systems with a plurality of target scanning spotscorresponding to postures in a quantity of the target value.

In an embodiment of the disclosure, the transformation of coordinatesystems includes a correspondence, a rotation angle, and a translationbetween a coordinate system of the vision sensor and a coordinate systemof the depth sensor.

Based on the above, the disclosure proposes a calibration method using aflat plate as the calibration target, and the accuracy of the obtainedtransformation of coordinate system between the vision sensor and thedepth sensor is ensured by the constraining area formed by the scanningspots generated by the depth sensor on the calibration board. Therefore,the calibration method improves the stability of the calibration resultsof the transformation of coordinate systems. Also, the calibrationmethod is not limited to the use of specific objects such ascheckerboard calibration boards and can be widely applied to calibrate atransformation of coordinate system between various depth sensors andvision sensors.

Other objectives, features and advantages of the present invention willbe further understood from the further technological features disclosedby the embodiments of the present invention wherein there are shown anddescribed preferred embodiments of this invention, simply by way ofillustration of modes best suited to carry out the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide a furtherunderstanding of the invention, and are incorporated in and constitute apart of this specification. The drawings illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention.

FIG. 1 is a block diagram illustrating a calibration system forcalibrating a visual coordinate system and a depth coordinate systemaccording to an embodiment of the disclosure.

FIG. 2 is a block diagram illustrating a calibration device according toan embodiment of the disclosure.

FIG. 3 is a flowchart illustrating a calibration method according to anembodiment of the disclosure.

FIG. 4 is a schematic view illustrating a transformation of coordinatesystem in a calibration system for calibrating a visual coordinatesystem and a depth coordinate system according to an embodiment of thedisclosure.

FIGS. 5A-5C are schematic views illustrating a method for obtaining atarget scanning spot according to an embodiment of the disclosure.

FIG. 6 is a flowchart illustrating a calibration method according toanother embodiment of the disclosure.

DESCRIPTION OF THE EMBODIMENTS

FIG. 1 is a block diagram illustrating a calibration system forcalibrating a visual coordinate system and a depth coordinate systemaccording to an embodiment of the disclosure. FIG. 2 is a block diagramillustrating a calibration device according to an embodiment of thedisclosure. Note that the embodiments in FIG. 1 and FIG. 2 are only forconvenience of description and are not intended to limit the disclosure.

Referring to FIG. 1, a calibration system 100 for calibrating a visualcoordinate system and a depth coordinate system includes a processor110, a vision sensor 120, a depth sensor 130, and a calibration board140. The processor 110 is coupled to the vision sensor 120 and the depthsensor 130, and the calibration board 140 is disposed within sensingranges of the vision sensor 120 and the depth sensor 130. In particular,the calibration board 140 is a flat plate in the embodiment of thedisclosure. In the embodiment, the calibration system 100 forcalibrating a visual coordinate system and a depth coordinate system maybe disposed in any scene such as an indoor environment, an outdoorenvironment, etc. so as to obtain a transformation of coordinate systemsbetween the vision sensor 120 and the depth sensor 130.

The processor 110 is coupled to the vision sensor 120 and the depthsensor 130 in a wireless manner or in a wired manner. In addition, theprocessor 110 is configured to obtain the transformation of coordinatesystems between the vision sensor 120 and the depth sensor 130. In thepresent embodiment, the processor 110 may be a general-purposeprocessor, a special-purpose processor, a conventional processor, adigital signal processor, multiple microprocessors, one or moremicroprocessors integrated with a digital signal processor core, acontroller, a microcontroller, an application specific integratedcircuit (ASIC), a field programmable gate array (FPGA), any other kindof integrated circuit, a state machine, a processor based on advancedreduced instruction set machine (ARM), and the like. In the embodiment,the processor 110 is configured to perform a calibration method of thedisclosure to obtain the transformation of coordinate systems betweenthe vision sensor 120 and the depth sensor 130.

The vision sensor 120 is configured to obtain an image of thecalibration board 140 and has an image capturing unit with a lens and aphotosensitive element. The photosensitive element is configured tosense the intensity of light passing through the lens to generateimages. The photosensitive element may be, for example, a charge coupleddevice (CCD) element, a complementary metal-oxide semiconductor (CMOS)element, or other elements, and the disclosure is not limited thereto.For example, the vision sensor 120 may be a camera such as a colorcamera, a monochrome camera, a near infrared camera, or the like.

The depth sensor 130 is configured to generate a plurality of scanningspots to detect depth information in front of it. The depth sensor 130is capable of calculating the depth information in front by activelyemitting light beams, ultrasound, lasers, etc. as signals. For example,the depth sensor 130 is capable of detecting the depth information withranging technologies, such as structured light, time-of-flight ranging,optical radar, and millimeter-wave radar. However, the ranging methodused by the depth sensor 130 is not limited in the disclosure, and depthsensors based on single-line laser radars, multi-line laser radars,dynamic laser points, area depth images, etc. may be applied to thecalibration system 100 for calibrating a visual coordinate system and adepth coordinate system in the embodiment.

In an embodiment of the disclosure, the processor 110 of the calibrationsystem 100 for calibrating a visual coordinate system and a depthcoordinate system may be implemented as a calibration device having acomputation processing function, but the disclosure is not limitedthereto. Specifically, referring to FIG. 2, in different embodiments, acalibration device 10 may be a smartphone, a tablet computer, a personalcomputer, a laptop computer, or other devices with computationprocessing functions, but the disclosure is not limited thereto. Asshown in FIG. 2, the calibration device 10 may include the processor 110and a storage circuit 20. The processor 110 is coupled to the storagecircuit 20. The storage circuit 20 is, for example, any type of fixed orremovable random access memory (RAM), read-only memory (ROM), flashmemory, hard disk, other similar devices, or a combination thereof.

In the embodiments of the disclosure, the calibration device 10 may beapplied to the calibration system 100 for calibrating a visualcoordinate system and a depth coordinate system in any scene. Thestorage circuit 20 of the calibration device 10 is configured to storethe image of the calibration board 140 obtained by the vision sensor 120and store the plurality of scanning spots generated by the depth sensor130 on the calibration board 140. The processor 110 is capable ofaccessing the image and the scanning spots recorded in the storagecircuit 20 and performing the calibration method proposed by thedisclosure. The details of the calibration method are as follows.

Referring to FIG. 3 and FIG. 4, FIG. 3 is a flowchart illustrating acalibration method according to an embodiment of the disclosure, andFIG. 4 is a schematic view illustrating a transformation of coordinatesystem in a calibration system for calibrating a visual coordinatesystem and a depth coordinate system according to an embodiment of thedisclosure. The calibration method of the embodiment may be performed bythe processor 110 of the calibration device 10 in FIG. 1 and FIG. 2.With reference to the elements shown in FIG. 1 and the schematic viewillustrating the transformation of coordinate systems of FIG. 4, thedetails of each step in FIG. 3 are described below.

First, before step S301, as shown in FIG. 4, the calibration board 140is disposed between the vision sensor 120 and the depth sensor 130, sothat a posture of the calibration board 140 is within a visible range(i.e., within the sensing range) of the vision sensor 120 and the depthsensor 130. Herein, the posture may include, for example, a position anda pose of the calibration board 140. Note that in the embodiment of thedisclosure, the transformation of coordinate systems (i.e., extrinsicparameters) between the vision sensor 120 and the depth sensor 130 to beobtained includes a correspondence, a rotation angle, and a translationbetween the coordinate system of the vision sensor 120 and thecoordinate system of the depth sensor 130. More specifically, an objecthas six degrees of freedom in space. The six degrees of freedom in spaceare the degrees of freedom of movement (i.e., the translation) along thethree orthogonal coordinate axes of X, Y, and Z and the degrees offreedom of rotation (i.e., the rotation angle) around the threecoordinate axes. Therefore, by obtaining the six degrees of freedom, theposition of the object can be determined.

In step S301, the processor 110 creates a first coordinate group of fourendpoints of the calibration board 140 in a world coordinate system(WCS). Specifically, the world coordinate system is a fixed coordinatesystem. The world coordinate system takes the world space or a modelspace as a whole and can be defined by the X-axis, Y-axis, and Z-axis.For example, a plane of the calibration board 140 in FIG. 4 coincideswith the X-Y plane, and it is assumed that a length of the calibrationboard 140 is h and a width of the calibration board 140 is w. Therefore,in step S301, by measuring the actual physical dimensions of thecalibration board 140, a coordinate W1[0,0,0], a coordinate W2[0,h,0], acoordinate W3[w,h,0], and a coordinate W4[w,0,0] of the four endpointsof the calibration board 140 in the world coordinate system areobtained, and the four coordinates in the world coordinate system arethe first coordinate group.

Next, in step S303, the processor 110 uses the image of the calibrationboard 140 obtained by the vision sensor 120 to create a secondcoordinate group of the four endpoints of the calibration board 140 in atwo-dimensional coordinate system according to the image of thecalibration board 140. For example, the Hough Transformation is usedherein to identify the features of the calibration board 140 in theimage. For example, the straight line features of the edges of thecalibration board 140 are identified, and the intersection points ofeach two lines are the four endpoints of the calibration board 140.Thus, a coordinate c1, a coordinate c2, a coordinate c3, and acoordinate c4 in a two-dimensional coordinate system according to theimage of the four endpoints of the calibration board 140 are detected.The four coordinates in the two-dimensional coordinate system are thesecond coordinate group.

After step S303, the processor 110 obtains a first transformationbetween the first coordinate group and the second coordinate groupaccording to a geometric correspondence between the first coordinategroup and the second coordinate group and the intrinsic parameter of thevision sensor 120. In the embodiment, the geometric correspondence is aPnP (perspective-n-point) transformation method for solving atransformation between a three-dimensional coordinate point and atwo-dimensional coordinate point, and the intrinsic parameter of thevision sensor 120 is, for example, focal length information includinginformation such as the focal length of the lens and the center positionof the lens. For example, the intrinsic parameter matrix of the visionsensor 120 with a lens focal length (fx, fy) and a lens center position(cy, cx) is K=[fx, 0, cx; 0, fy, cy; 0, 0, 1]. Moreover, the intrinsicparameters and the intrinsic parameter matrix can be obtained through acalibration procedure for calibrating lens distortion or deformationbefore step S301. The first coordinate group in the world coordinatesystem and the second coordinate group in the two-dimensional coordinatesystem conform to the geometric correspondence of the PnP transformationmethod for 3D-2D correspondence matching. Therefore, by substituting theintrinsic parameter matrix K into the PnP transformation method, thetransformation π_(i) ^(W→C) (i.e., the first transformation π_(i)^(W→C)) shown in FIG. 4 can be obtained. The index value i is a positiveinteger greater than 0 and represents the posture of the i-thcalibration board 140 used in the calibration method. W→C represents thetransformation from the world coordinate system to the visual coordinatesystem (camera coordinate system).

In step S305, the processor 110 creates a third coordinate group of thefour endpoints of the calibration board 140 corresponding to the visionsensor 120 in the three-dimensional coordinate system according to thefirst coordinate group and the second coordinate group. In the step, theprocessor 110 transforms the coordinate W1[0,0,0], the coordinateW2[0,h,0], the coordinate W3[w,h,0], and the coordinate W4[w,0,0] of thefirst coordinate group in the world coordinate system to a coordinateC1i, a coordinate C2i, a coordinate C3i, and a coordinate C4i (notshown) in the three-dimensional coordinate system, respectively,according to the first transformation π_(i) ^(W→C). The four coordinatesin the three-dimensional coordinate system are the third coordinategroup corresponding to the vision sensor 120.

In step S307, the processor 110 transforms the third coordinate group ofthe vision sensor to a fourth coordinate group corresponding to thedepth sensor 130. The processor 110 calculates the transformation ofcoordinate systems between the vision sensor 120 and the depth sensor130 according to at least three target scanning spots. Specifically, theprocessor 110 transforms the third coordinate group to the fourthcoordinate group according to a transformation π^(C→D) (i.e., a secondtransformation π^(C→D)). C→D represents the transformation from thevisual coordinate system to the depth coordinate system. That is, thecoordinate C1i, the coordinate C2i, the coordinate C3i, and thecoordinate C4i in the three-dimensional coordinate system aretransformed to a coordinate P1i, a coordinate P2i, a coordinate P3i, anda coordinate P4i corresponding to the coordinate systems of the depthsensor (i.e., a coordinate P1, a coordinate P2, a coordinate P3, and acoordinate P4 as shown in FIG. 4). Similarly, the index value i is theposture of the i-th calibration board 140 used in the calibrationmethod. The four coordinates in the coordinate system of the depthsensor are the fourth coordinate group. In particular, the secondtransformation π^(C→D) is the correspondence between the coordinatesystem of the vision sensor and the coordinate system of the depthsensor. Therefore, the transformation π^(C→D) is the transformation ofcoordinate systems to be obtained by the calibration method of thedisclosure.

Furthermore, in step S307 of obtaining the transformation of thecoordinate systems between the vision sensor 120 and the depth sensor130 according to the fourth coordinate group and the at least threetarget scanning spots of the depth sensor 130 on the calibration board140, the method for obtaining the target scanning spots is illustratedwith reference to FIG. 5A to FIG. 5C. FIG. 5A to FIG. 5C are schematicviews illustrating the method for obtaining a target scanning spotaccording to an embodiment of the disclosure.

Specifically, the depth sensor 130 generates a plurality of scanningspots Pi(j), j=1, . . . , Ni in the calibration system 100 forcalibrating a visual coordinate system and a depth coordinate system. Nrepresents the count of scanning spots generated by the depth sensor130, and the index value i represents the posture of the i-thcalibration board used in the calibration method. The processor 110calculates the second transformation (i.e., the transformation ofcoordinate systems) and ensures that the target scanning spots fallingon the calibration board 140 falls within a quadrilateral formed by thefour endpoints of the calibration board 140. In the embodiment of thedisclosure, the scanning spots and the four endpoints of the calibrationboard 140 are used to form a constraining area. The constraint conditionof the constraining area is that an area enclosed by the four endpointsof the calibration board 140 must equal to a sum of areas of the fourtriangles formed by any scanning spot Pi (j) and the four endpoints.

More specifically, referring to FIG. 5A, to quickly obtain the areaenclosed by the four endpoints of the calibration board 140, in theembodiment of the disclosure, the quadrilateral formed by the fourendpoints of the calibration board 140 is regarded as two triangles, andthe areas of two triangles are obtained, respectively. As shown in FIG.5A, the area of a triangle with three endpoints p1, p2, and p3 and sidelengths a, b, and c can be quickly obtained through the followingHeron's formula (1).A(p1,p2,p3)=(s(s−a)(s−b)(s−c))^(1/2) ,s=(a+b+c)/2  Formula (1)

Next, referring to FIG. 5B and FIG. 5C, the area of the quadrilateralformed by the four endpoints p1, p2, p3, and p4 in FIG. 5B and FIG. 5Ccan be obtained by calculating the sum of the areas of the two triangles(that is, obtained by Formula (1) above) and expressed as A(p1, p2,p3)+A(p1, p3, p4). In the case of FIG. 5B, if any point p falls withinthe quadrilateral formed by the four endpoints p1, p2, p3, and p4, thenas shown in Formula (2), the area of the quadrilateral formed by thefour endpoints p1, p2, p3, and p4 is equal to the sum of the areas ofthe four triangles formed by the point p and the endpoints p1, p2, p3,and p4, respectively.A(p1,p2,p3)+A(p1,p3,p4)=A(p1,p2,p)+A(p2,p3,p)+A(p3,p4,p)+A(p1,p4,p)  Formula (2)

In addition, in the case of FIG. 5C, if any point p′ falls outside thequadrilateral formed by the four endpoints p1, p2, p3, and p4, the sumof the areas of the four triangles formed by the point p′ and theendpoints p1, p2, p3, and p4 is greater than the area of thequadrilateral formed by the four endpoints p1, p2, p3, and p4 as shownin Formula (3).A(p1,p2,p3)+A(p1,p3,p4)<A(p1,p2,p′)+A(p2,p3,p′)+A(p3,p4,p′)+A(p1,p4,p′)  Formula (3)

Based on the method for obtaining the target scanning spot, in theembodiment of the disclosure, as shown in FIG. 4, the area of thequadrilateral formed by the four endpoints of the calibration board 140obtained by the processor 110 is expressed as A(p1, p2, p3)+A(p1, p2,p4). The sum of the areas of the four triangles respectively formed byany point P (not shown) and the four endpoints of the quadrilateral isexpressed as A(p1, p2, p)+A(p2, P3, P)+A(P3, P4, P)+A(P1, P4, P), andthe following target Formula (4) can be defined.Cover(P1,P2,P3,P4,P)={[A(P1,P2,P3)+A(P1,P2,P4)]−[A(P1,P2,P)+A(P2,P3,P)+A(P3,P4,P)+A(P1,P4,P)]}²  Formula(4)

Taking the presence of noise in the vision sensor and the depth sensorinto consideration, the second transformation (i.e., the transformationof coordinate systems) to be obtained by the processor 110 is configuredso that a difference between the sum of the areas of the four trianglesformed by the target scanning spot and the four endpoints and the areaof the quadrilateral is less than an error value. In other words, thearea difference between the two areas (i.e., A(P1, P2, P3)+A(P1, P3, P4)and A(P1, P2, P)+A(P2, P3, P)+A(P3, P4, P)+A(P1, P4, P)) is favorably assmall as possible, and that is, the value of Cover (P1, P2, P3, P4, P)is favorably as small as possible. Note that in the embodiment of thedisclosure which uses a plurality of calibration boards for calibration,the target formula is also expressed as Cover (P1i, P2i, P3i, P4i,Pi(j)), and the index value i represents the posture of the i-thcalibration board used in the calibration method.

Note that the constraint of the method for obtaining target scanningspots with the constraining area in the disclosure is stronger than theconstraint of the conventional method between laser points falling onthe calibration board and the normal vectors on the calibration board,so it is possible to ensure that the obtained scanning spot falls withinthe quadrilateral formed by the calibration board 140. Also, the methodof the disclosure is not limited to the form of scanning spots.Accordingly, the method of the disclosure can be widely applied to depthsensors such as single-line laser radars, multi-line laser radars,dynamic laser points, area depth images, and the like.

Referring to step S307 in FIG. 3, after the processor 110 obtains atleast three target scanning spots, the second transformation (i.e., thetransformation of coordinate systems) between the vision sensor 120 andthe depth sensor 130 is obtained according to the four endpoints (i.e.,the coordinate W1[0,0,0], the coordinate W2[0,h,0], the coordinateW3[w,h,0], and the coordinate W4[w,0,0]) in the first coordinate groupobtained in step S301 to step S305, the first transformation π_(i)^(W→C), and the at least three target scanning spots Pi(j). Based on theabove, in the embodiment of the disclosure, first the processor 110transforms the third coordinate group corresponding to the vision sensor120 to the fourth coordinate group corresponding to the depth sensor 130based on an unknown second transformation, then the at least threetarget scanning spots are obtained through the constraining area formedby the scanning spot and the fourth coordinate group, and according tothe at least three target scanning spots, at least three formulas arelisted to solve the second transformation to obtain the transformationof coordinate systems between the vision sensor 120 and the depth sensor130.

In particular, in the embodiment of the disclosure, to improve theaccuracy of the transformation of coordinate systems between the visionsensor 120 and the depth sensor 130, step S301 to step S307 of thecalibration method in FIG. 3 are performed on a plurality of calibrationboards with different postures to obtain an optimal solution to theextrinsic parameters. FIG. 6 is a flowchart illustrating a calibrationmethod according to another embodiment of the disclosure. The steps ofperforming the calibration method of the disclosure on a plurality ofcalibration boards are described below with reference to FIG. 6.

Referring to FIG. 6, step S301 to step S307 in FIG. 6 are the same asstep S301 to step S307 in FIG. 3. Therefore, these steps are notrepeatedly described herein. For example, in the embodiment of thedisclosure, after the processor 110 obtains the at least three targetscanning spots corresponding to the calibration board 140 having a firstposture (in step S307), the processor 110 obtains another posture (i.e.,a second posture) of the calibration board 140 in step S601 anddetermines whether a posture count value is greater than or equal to atarget value. For example, in the embodiment, assuming that the targetvalue is 30 and the posture count value corresponding to the calibrationboard 140 having the first posture is 1, the processor 110 determinesthat the posture count value is less than the target value and countsthe posture count value in step S603. That is, the current posture countvalue is incremented by 1 to be 2, and step S301 is performed again onthe obtained calibration board 140 having the second posture. That is,in the embodiment, since the target value is 30, the processor 110collects posture information of at least 30 calibration boards withdifferent postures and uses it as an object to perform the calibrationmethod of the disclosure. Note that the disclosure does not limit thetarget value. For example, in another embodiment, the target value maybe set to be greater than 30 or less than 30, and the target value mayalso be set according to the accuracy of the calibration result andactual needs. For example, in an embodiment, the more postures of thecalibration boards, the smaller the error value of the extrinsicparameters to be obtained. Alternatively, in another embodiment, thetarget value may also be set according to the calibration system 100 forcalibrating a visual coordinate system and a depth coordinate system indifferent scenes.

Furthermore, when the processor 110 determines that the posture countvalue is greater than or equal to the target value in step S601, theprocessor 110 solves the transformation of coordinate systems with aplurality of target scanning spots corresponding to postures in thequantity of the target value in step S605. For example, in theaforementioned embodiment, the processor 110 solves the transformationof coordinate systems with a plurality of target scanning spotscorresponding to 30 postures. Specifically, the processor 110 obtainsthe transformation of coordinate systems between the vision sensor 120and the depth sensor 130 according to the four endpoints (i.e., thecoordinate W1[0,0,0], the coordinate W2[0,h,0], the coordinateW3[w,h,0], and the coordinate W4[w,0,0]) in the first coordinate group,the first transformation π_(i) ^(W→C), and the target scanning spotsPi(j) corresponding to the i postures (e.g., i=30). For example, in theembodiment of the disclosure, the Levenberg-Marquardt algorithm isapplied to provide a nonlinear optimized numerical solution to obtain acorrespondence (π^(C→D))*, a rotation angle r*=(r_(α)*, r_(β)*, r_(γ)*),and a translation t*=(t_(x)*, t_(y)*, t_(z)*) from the coordinate systemof the vision sensor 120 to the coordinate system of the depth sensor130. The non-linear formula of the obtained correspondence (π^(C→D))* isshown in the following Formula (5).(π^(C→D))*=arg min Σ_(i)Σ_(j) ^(N) ^(i) Cover(π^(C→D)·π_(i) ^(W→C)·W1,π^(C→D)·π_(i) ^(W→C) ·W2,π^(C→D)·π_(i) ^(W→C) ·W3,π^(C→D)·π_(i)^(W→C) ·W4,Pi(j))  Formula (5)

Note that in another embodiment of the disclosure, the processor 110 iscapable of further transforming the rotation angle r*=(r_(α)*, r_(β)*,r_(γ)*) to a rotation matrix R* according to Rodrigues' rotationformula, and thereby the correspondence Formula (6) of thetransformation from the coordinate system of the depth sensor 130 to thecoordinate system of the vision sensor 120 is further obtained accordingto the rotation matrix R*.

$\begin{matrix}{\left( \pi^{D\rightarrow C} \right)^{*} = \begin{bmatrix}R^{*T} & {{- R^{*T}}t} \\0^{T} & 1\end{bmatrix}} & {{Formula}\mspace{14mu}(6)}\end{matrix}$

Based on the above, the calibration system for calibrating a visualcoordinate system and a depth coordinate system, the calibration method,and the calibration device of the disclosure do not limit the type ofdepth sensors or the type of calibrated objects used. In addition, byusing a common flat plate as the calibration board, the coordinates ofthe four endpoints of the calibration board are transformed between theworld coordinate system, the two-dimensional and three-dimensionalcoordinate systems of the vision sensor, and the coordinate system ofthe depth sensor to obtain the calibration results of the transformationof the coordinate systems (i.e., the extrinsic parameters) between thevision sensor and the depth sensor. Accordingly, the calibration methodfor obtaining the extrinsic parameters is more flexible, and may bewidely used in depth sensors, such as a single-line laser radar, amulti-line laser radar, a dynamic laser point, an area depth image, andthe like. As a result, the disclosure effectively improves theperformance of the calibration process, the utility of the calibrationmethod, and the stability of the calibration results. In addition, thecalibration method of the disclosure can ensure that the obtainedscanning spots of the depth sensor fall within the four endpoints of thecalibration board through the constraining area. Therefore, the accuracyof the calibration results is improved by obtaining the extrinsicparameter with target scanning spots that fall within the four endpointsof the calibration board. Also, errors of the calibration results arereduced with the method of the disclosure for calibrating a plurality ofcalibration boards with different postures as calibration objects.Accordingly, it is possible to ensure the calibration quality of theextrinsic parameter in a calibration system for calibrating a visualcoordinate system and a depth coordinate system.

The foregoing description of the preferred embodiments of the inventionhas been presented for purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseform or to exemplary embodiments disclosed. Accordingly, the foregoingdescription should be regarded as illustrative rather than restrictive.Obviously, many modifications and variations will be apparent topractitioners skilled in this art. The embodiments are chosen anddescribed in order to best explain the principles of the invention andits best mode practical application, thereby to enable persons skilledin the art to understand the invention for various embodiments and withvarious modifications as are suited to the particular use orimplementation contemplated. It is intended that the scope of theinvention be defined by the claims appended hereto and their equivalentsin which all terms are meant in their broadest reasonable sense unlessotherwise indicated. Therefore, the term “the invention”, “the presentinvention” or the like does not necessarily limit the claim scope to aspecific embodiment, and the reference to particularly preferredexemplary embodiments of the invention does not imply a limitation onthe invention, and no such limitation is to be inferred. The inventionis limited only by the spirit and scope of the appended claims.Moreover, these claims may refer to use “first”, “second”, etc.following with noun or element. Such terms should be understood as anomenclature and should not be construed as giving the limitation on thenumber of the elements modified by such nomenclature unless specificnumber has been given. The abstract of the disclosure is provided tocomply with the rules requiring an abstract, which will allow a searcherto quickly ascertain the subject matter of the technical disclosure ofany patent issued from this disclosure. It is submitted with theunderstanding that it will not be used to interpret or limit the scopeor meaning of the claims. Any advantages and benefits described may notapply to all embodiments of the invention. It should be appreciated thatvariations may be made in the embodiments described by persons skilledin the art without departing from the scope of the present invention asdefined by the following claims. Moreover, no element and component inthe present disclosure is intended to be dedicated to the publicregardless of whether the element or component is explicitly recited inthe following claims.

What is claimed is:
 1. A calibration method for obtaining atransformation of coordinate systems between a vision sensor and a depthsensor, wherein the calibration method comprises: (a) creating a firstcoordinate group of four endpoints of a calibration board in a worldcoordinate system; (b) obtaining an image of the calibration board bythe vision sensor and creating a second coordinate group of the fourendpoints of the calibration board in a two-dimensional coordinatesystem; (c) creating a third coordinate group of the four endpoints ofthe calibration board corresponding to the vision sensor in athree-dimensional coordinate system according to the first coordinategroup and the second coordinate group; and (d) transforming the thirdcoordinate group to a fourth coordinate group corresponding to the depthsensor to obtain the transformation of coordinate systems according toat least three target scanning spots generated by the depth sensor. 2.The calibration method according to claim 1, wherein after the step ofcreating the second coordinate group of the four endpoints of thecalibration board in the two-dimensional coordinate system, thecalibration method further comprises: obtaining a first transformationbetween the first coordinate group and the second coordinate groupaccording to a geometric correspondence between the first coordinategroup and the second coordinate group and an intrinsic parameter of thevision sensor, wherein the step of creating the third coordinate groupof the four endpoints of the calibration board in the three-dimensionalcoordinate system according to the first coordinate group and the secondcoordinate group comprises: transforming the first coordinate group tothe third coordinate group according to the first transformation.
 3. Thecalibration method according to claim 1, wherein the step oftransforming the third coordinate group to the fourth coordinate groupcorresponding to the depth sensor comprises: transforming the thirdcoordinate group to the fourth coordinate group according to a secondtransformation, wherein the second transformation is a correspondencebetween a coordinate system of the vision sensor and a coordinate systemof the depth sensor.
 4. The calibration method according to claim 3,wherein the at least three target scanning spots are located in aquadrilateral formed by the four endpoints in the fourth coordinategroup.
 5. The calibration method according to claim 4, wherein the stepof transforming the third coordinate group to the fourth coordinategroup corresponding to the depth sensor to obtain the transformation ofcoordinate systems according to the at least three target scanning spotsgenerated by the depth sensor comprises: calculating a sum of areas offour triangles formed by scanning spots generated by the depth sensorand the four endpoints in the fourth coordinate group; and solving thesecond transformation by taking, as the target scanning spots, scanningspots of which a difference between the sum of the areas of the fourtriangles and an area of the quadrilateral is less than an error value.6. The calibration method according to claim 2, wherein the calibrationboard is a flat plate, a posture of the calibration board is within avisible range of the vision sensor and the depth sensor, and thecalibration method further comprises: (e) obtaining another posture ofthe calibration board and determining whether a posture count value isgreater than or equal to a target value, wherein if the posture countvalue is greater than or equal to the target value, step (f) isperformed, and if the posture count value is less than the target value,the posture count value is counted, and step (a) is re-performed on thecalibration board with the another posture; and (f) solving thetransformation of coordinate systems with a plurality of target scanningspots corresponding to postures in a quantity of the target value. 7.The calibration method according to claim 6, wherein the transformationof coordinate systems comprises a correspondence, a rotation angle, anda translation between a coordinate system of the vision sensor and acoordinate system of the depth sensor.
 8. A calibration system forcalibrating a visual coordinate system and a depth coordinate system,comprising: a vision sensor configured to obtain an image of acalibration board; a depth sensor configured to generate a plurality ofscanning spots; and a processor coupled to the vision sensor and thedepth sensor to obtain a transformation of coordinate systems betweenthe vision sensor and the depth sensor, wherein the processor isconfigured to: (a) create a first coordinate group of four endpoints ofthe calibration board in a world coordinate system; (b) create a secondcoordinate group of the four endpoints of the calibration board in atwo-dimensional coordinate system according to the image of thecalibration board; (c) create a third coordinate group of the fourendpoints of the calibration board corresponding to the vision sensor ina three-dimensional coordinate system according to the first coordinategroup and the second coordinate group; and (d) transform the thirdcoordinate group to a fourth coordinate group corresponding to the depthsensor to obtain the transformation of coordinate systems according toat least three target scanning spots.
 9. The calibration system forcalibrating the visual coordinate system and the depth coordinate systemaccording to claim 8, wherein after the operation of creating the secondcoordinate group of the four endpoints of the calibration board in thetwo-dimensional coordinate system, the processor is configured to:obtain a first transformation between the first coordinate group and thesecond coordinate group according to a geometric correspondence betweenthe first coordinate group and the second coordinate group and anintrinsic parameter of the vision sensor, wherein in the operation ofcreating the third coordinate group of the four endpoints of thecalibration board in the three-dimensional coordinate system accordingto the first coordinate group and the second coordinate group, theprocessor is configured to: transform the first coordinate group to thethird coordinate group according to the first transformation.
 10. Thecalibration system for calibrating the visual coordinate system and thedepth coordinate system according to claim 8, wherein in the operationof transforming the third coordinate group to the fourth coordinategroup corresponding to the depth sensor, the processor is configured to:transform the third coordinate group to the fourth coordinate groupaccording to a second transformation, wherein the second transformationis a correspondence between a coordinate system of the vision sensor anda coordinate system of the depth sensor.
 11. The calibration system forcalibrating the visual coordinate system and the depth coordinate systemaccording to claim 10, wherein the at least three target scanning spotsare located in a quadrilateral formed by the four endpoints in thefourth coordinate group.
 12. The calibration system for calibrating thevisual coordinate system and the depth coordinate system according toclaim 11, wherein in the operation of transforming the third coordinategroup to the fourth coordinate group corresponding to the depth sensorto obtain the transformation of coordinate systems according to the atleast three target scanning spots generated by the depth senor on thecalibration board, the processor is configured to: calculate a sum ofareas of four triangles formed by the scanning spots and the fourendpoints in the fourth coordinate group; and solve the secondtransformation by taking, as the target scanning spots, scanning spotsof which a difference between the sum of the areas of the four trianglesand an area of the quadrilateral is less than an error value.
 13. Thecalibration system for calibrating the visual coordinate system and thedepth coordinate system according to claim 9, wherein the calibrationboard is a flat plate, and a posture of the calibration board is withina visible range of the vision sensor and the depth sensor, wherein theprocessor is further configured to: (e) obtain another posture of thecalibration board and determine whether a posture count value is greaterthan or equal to a target value, wherein if the posture count value isgreater than or equal to the target value, operation (f) is performed,and if the posture count value is less than the target value, theposture count value is counted, and operation (a) is re-performed on thecalibration board with the another posture; and (f) solve thetransformation of coordinate systems with a plurality of target scanningspots corresponding to postures in a quantity of the target value. 14.The calibration system for calibrating the visual coordinate system andthe depth coordinate system according to claim 13, wherein thetransformation of coordinate systems comprises a correspondence, arotation angle, and a translation between a coordinate system of thevision sensor and a coordinate system of the depth sensor.
 15. Acalibration device, comprising a storage circuit storing images of acalibration board obtained by a vision sensor and storing a plurality ofscanning spots generated by a depth sensor on the calibration board; anda processor coupled to the storage circuit and accessing the images andthe scanning spots to: (a) create a first coordinate group of fourendpoints of the calibration board in a world coordinate system; (b)create a second coordinate group of the four endpoints of thecalibration board in a two-dimensional coordinate systems according tothe images of the calibration board; (c) create a third coordinate groupof the four endpoints of the calibration board corresponding to thevision sensor in a three-dimensional coordinate system according to thefirst coordinate group and the second coordinate group; and (d)transform the third coordinate group to a fourth coordinate groupcorresponding to a depth sensor to obtain a transformation of coordinatesystems according to at least three target scanning spots.
 16. Thecalibration device according to claim 15, wherein after the operation ofcreating the second coordinate group of the four endpoints of thecalibration board in the two-dimensional coordinate system, theprocessor is configured to: obtain a first transformation between thefirst coordinate group and the second coordinate group according to ageometric correspondence between the first coordinate group and thesecond coordinate group and an intrinsic parameter of the vision sensor,wherein in the operation of creating the third coordinate group of thefour endpoints of the calibration board in the three-dimensionalcoordinate system according to the first coordinate group and the secondcoordinate group, the processor is configured to: transform the firstcoordinate group to the third coordinate group according to the firsttransformation.
 17. The calibration device according to claim 15,wherein in the operation of transforming the third coordinate group tothe fourth coordinate group corresponding to the depth sensor, theprocessor is configured to: transform the third coordinate group to thefourth coordinate group according to a second transformation, whereinthe second transformation is a correspondence between a coordinatesystem of the vision sensor and a coordinate system of the depth sensor.18. The calibration device according to claim 17, wherein the at leastthree target scanning spots are located in a quadrilateral formed by thefour endpoints in the fourth coordinate group.
 19. The calibrationdevice according to claim 18, wherein in the operation of transformingthe third coordinate group to the fourth coordinate group correspondingto the depth sensor to obtain the transformation of coordinate systemsaccording to the at least three target scanning spots generated by thedepth sensor on the calibration board, the processor is configured to:calculate a sum of areas of four triangles formed by the scanning spotsand the four endpoints in the fourth coordinate group; and solve thesecond transformation by taking, as the target scanning spots, scanningspots of which a difference between the sum of the areas of the fourtriangles and the area of the quadrilateral is less than an error value.20. The calibration device according to claim 16, wherein thecalibration board is a flat plate, and a posture of the calibrationboard is within a visible range of the vision sensor and the depthsensor, wherein the processor is further configured to: (e) obtainanother posture of the calibration board and determine whether a posturecount value is greater than or equal to a target value, wherein if theposture count value is greater than or equal to the target value,operation (f) is performed, and if the posture count value is less thanthe target value, the posture count value is counted, and operation (a)is re-performed on the calibration board with the another posture; and(f) solve the transformation of coordinate systems with a plurality oftarget scanning spots corresponding to postures in a quantity of thetarget value.
 21. The calibration device according to claim 20, whereinthe transformation of coordinate systems comprises a correspondence, arotation angle, and a translation between a coordinate system of thevision sensor and a coordinate system of the depth sensor.